CASCADE ERROR PROJECTION LEARNING TllEORY

Tuan A. Duong”+, Allen R. Stubberud+, and Taher Daud* “ Center for Space Microelectronics Technology Jet Propulsion Laboratory, California Institute of Technology Pasadena, CA 91109 + Department of Electrical Engineering, University of California, Irvine Irvine, CA 92717 Abs[ract: Ifie cascade correlation based neural network learning algorithm has drwwvt a lot of attention because of its enhanced learning capability. It overcomes significant drawbacks of error backpropagalion (EBP) in that (1) it is no longer constrained 10 a fixed architecture via a preallocation of the number of hidden units, (2) it features seiec(ive weigh( [raining as opposed to EBP’s global weight training. In addition, jiJom a hardware implementation perspective, nel works based on the cascade correlation algorithm require sign ijcantly less complex synaptic weight circuitry than those reqliired by EBP. We present a rnathernalical analysis for a new scheme termed Cascade Error Projection (CEP) and show tha( (he same is also applicable to Cascade Correlation. In CEP, it is shown that there exis!s, al least, a non zero set of weights which is calculaledfiom afine space and that convergence is assured because the net work salisfies the Liapunov criteria, iti added hidden units domain rather than in the time domain. The CEP technique is faster to execute because part of the weigh(s are dcterrnin is[ical[y obtained, and the learning of weights fiorn the inputs to each added hidden unit is perjorrned as a single layer percepiron learning with previously obtained weights frozen. In addition, Ihc initial weights s[arl oul with a zero value for e~’ery newly added unit, and a single hidden unit is applied inslead of using a pool of candidate hidden units as for cascade correlation, lfiereby,